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We pres_ent a study of the decays B° ^ D°K*° and B° D°K*° with K*° K+w'. The D° 

and the mesons arc reconstructed in the final states / = K^tv~, K'^tt~tv^, tv~ tv'^ it~ and 
their charge conjugates. Using a sample of 465 million BB pairs collected with the BABAR detector 
at the PEP-II asymmetric-energy e^'^e^ collider at SLAC, we measure the ratio Rads = — > 
lf]DK*°) + r{B° [f]DK*'')]/[T(B° ^ [f]DK*°) + r{B° [f]DK*°)] for the three final states. 
We do not find significant evidence for a signal and set the following limits at 95% probability: 
Rads{Ktv) < 0.244, Rads{Ktvtt°) < O.fSf and RADsiKTTTnr) < 0.391. From the combination of 
these three results, we find that the ratio rs between the b u and the b ^ c amplitudes lies in 
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Various methods have beenproposed to determine the 
Unitarity Triangle angle 7 [U, S Q of the Cabibbo- 
Kobayashi-Maskawa (CKM) quark mixing matrix [3| us- 
ing B- d(.*)Ok(*')- decays, where the symbol D(*'>° 
indicates either a D'-*'^ or a meson. A B meson 

can decay into a D^*^^ K^*^^ final state via a 6 — > c or a 
& — > u process. CP violation may occur due to interfer- 
ence between the amplitudes when the and 
decay to the same final state. These processes are thus 
sensitive to 7 = arg{- l/*^K<i/K1.^cd}- The sensitivity 
to 7 is proportional to the ratio between the 6 — s- m and 
& — > c transition amplitudes {tb), which depends on the 
B decay channel and needs to be determined experimen- 
tally 

In this paper we consider an alternative approach, 
based on neutral B mesons, which is similar to the 
ADS method [1] originally proposed for charged B~ ^ 
f)i*)Oj^{*)- (decays. We consider the decay channel B^ 
D^K*^ with K*^ K^TT~ (charge conjugate processes 
are assumed throughout the paper and K*^ refers to the 
K* (892)°) . This final state can be reached through b c 
and b u processes as shown in Fig. [TJ The flavor of 




FIG. 1: Feynman diagrams for 5° D°K*° (left, b ^ c 
transition) and — > D^K**^ (right, h ^ u transition). 

the B meson is identified by the charge of the kaon pro- 
duced in the K*^ decay. The neutral D mesons are re- 
constructed in three final states, / = if+Tr", _ftr+7r~7r°, 
K+tt^tt+t:-. We search for 5° -^_[f]D[K+Tr-]K'» 
events, where the CKM-favored 5° — > D"K*° decay, fol- 
lowed by the doubly Cabibbo-suppressed ^ J de- 
cay, interferes with the CKM-suppressed 5° ^_D°i^*° 
decay, followed by the Cabibbo-favored D° ^ f decay. 
These are called "opposite-sign" events because the two 
kaons in the final state have opposite charges. We also 
reconstruct a larger sample of "same-sign" events, which 
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mainly arise from CKM-favored B'^ D^K*^ decays 
followed by Cabibbo-favored Z?'^ f decays. 

In order to reduce the systematic uncertainties, we 
measure ratios of decay rates: 



Rads = 



r(s" 



A 



ADS 



t(b" 
r(B" 



T{BO 

r{B° 



[f]DK*°) 



-*0 



r{B ^ [f]DK ')+T{B" ^[f]DK*o) 



(1) 
(2) 



where Rads is the ratio between opposite- and same-sign 
events. 

The resonance has a natural width (50 MeV/c^) 
that is larger than the experimental resolution. This in- 
troduces a phase difference between the various ampli- 
tudes. We therefore introduce effective variables 7-5, fc, 
and ^5 obtained by integrating over the region of 
the B° D°K+TT- Dalitz plot dominated by the 
resonance, defined as follows: 



' s 



ke 



r(5° ^ D^'K+Tr-) _ JdpAljp) 
r(B" ^ D"K+TT-) ^ Jdp A2(p) ' 

~ sJjdpA^M JdpAlip)' 



(3) 
(4) 



From their definition, < fc < 1 and (5s € [0,27r]. The 
amplitudes for the b c and b ^ u transitions, Ac(p) 
and Au(jp), are real and positive and 5{p) is the rela- 
tive strong phase. The variable p indicates the position 
in the D'^K^tt~ Dalitz plot. The parameter k accounts 
for contributions, in the K*'^ mass region, of higher-mass 
resonances. In the case of a two-body B decay, and 
Ss become tb — A^/Ac and Sb (the strong phase dif- 
ference between A^ and Ac) with k = I. As shown 
in ^ , the distribution of k can be obtained by simulation 
studies based on realistic models for the different reso- 
nance contributions to the decays of neutral B mesons 
into D^K^TT^ final states. When considering the region 
in the B^ D^K'^tt" Dalitz plane where the invariant 
mass of the kaon and the pion is within 48 MeV/c^ of the 
nominal K*^ mass [7], the distribution of k is narrow, 
and is centered at 0.95 with a root-mean-square width of 
0.03. 

Because of CKM factors and the fact that both di- 
agrams in Fig. [T] are color-suppressed, the average am- 
plitude ratio rs in B^ D'^K*^ is expected to be 
of order 0.3, larger than the analogous ratio for the 
charged B~ — > I)^{*)k^*)- decays, which is of order 
0.1 [1, [1]. This implies better sensitivity to 7 for the 
same number of events, an expectation that applies to 
aU B° £)0(*)^(*)o (decays, and that motivates the use 
of neutral B meson decays to determine 7. Currently, 
the experimental knowledge of rg [1, [l3| is rs < 0.54 at 
95% probability. 

The ratios Rads a-nd Aads are related to rg, 7, k and 
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6s through the following relations: 

Rads rl + r% + 2kkDrsrD cos J cos{Ss + Sd), (5) 
Aads = 2kkDrsrD sin J am{5s + Sd)/ Rads , (6) 

where 

2 ^ r(D° ^ /) ^ JdmAlcsim) 

\J S dm Al;p{m) J dm Al,(jg{m) 

with < fc^i < 1, i5_D € [0, 27r], AcF{m) and ^13(75(771) 
the magnitudes of the Cabibbo-favored and the doubly- 
Cabibbo-suppressed amplitudes, (5(m) the relative strong 
phase, and the variable m the position in the D Dalitz 
plot. In the case of a two-body D decay, ko — 1, 
is the ratio between the doubly-Cabibbo-suppressed and 
the Cabibbo-favored decay amplitudes and Sd is the rel- 
ative strong phase. 

Determining rs, 7 and ds from the measurements of 
Rads and A^^s, with the factor k fixed, requires knowl- 
edge of the parameters {ko, rjj, 5d), which depend on 
the specific neutral D meson final states. The ratios r^) 
for the three D decay modes have been measured [7[ , as 
has the strong phase So for the Kn mode In addi- 
tion, experimental information is available on kjj and S u 
for the Kn-K^ and Ktitttt modes The smallness of 

the rD ratios implies good sensitivity to rs from a mea- 
surement of Rads- For the same reason, and since, with 
the present statistics, the asymmetries Aads cannot be 
extracted from data, the sensitivity to 7 is reduced. The 
aim of this analysis is therefore the measurement of rs- 
In the future, good knowledge of all the rjj, ko and 
parameters, and a precise measurement of the Rads ra- 
tios for the three channels, will allow 7 and Ss to be 
determined from this method as well. 

The results presented here are obtained with 423 fb^^ 
of data collected at the T(45) resonance with the BaBAR 
detector at the PEP-II e+e" colhder at SLAC cor- 
responding to 465 million BB events. An additional 
"off-resonance" data sample of 41.3 fb~^, collected at 
a center-of-mass (CM) energy 40 MeV below the T{AS) 
resonance, is used to study backgrounds from continuum 
events, e+e~ ^ qq [q — u, d,s, or c). The BaBAR detec- 
tor is described elsewhere |14| . 

The event selection is based on studies of off-resonance 
data and Monte Carlo (MC) simulations of continuum 
and e+e" T{4S) BB events. All the selection crite- 
ria are optimised by maximising the function S/ \/ S + B 
on opposite-sign events, where S and B are the expected 
numbers of opposite-sign signal and background events, 
respectively. 

The neutral D mesons are reconstructed from a 
charged kaon and one or three charged pions and, 
in the Kttt:^ mode, a neutral pion. The tt" candi- 
dates are reconstructed from pairs of photon candidates, 
each with energy greater than 70 MeV, total energy 



greater than 200 MeV and invariant mass in the inter- 
val [118, 145] MeV/c^ The 7r° candidate's mass is subse- 
quently constrained to its nominal value Q ■ 

The invariant mass of the particles used to reconstruct 
the D is required to lie within 14 MeV/c^ (~ 1.9cr), 20 
MeV/c^ (~ 1.5cr) and 9 MeV/c^ (~ 1.6cr) of the nomi- 
nal mass, for the Kt:, Kttt:^ and Kmnr modes, re- 
spectively. For the Kmnr mode we also require that the 
tracks originate from a single vertex with a probability 
greater than 0.1%. 

The tracks used to reconstruct the K*'^ are constrained 
to originate from a common vertex and their invariant 
mass is required to lie within 48 MeV/ of the nominal 
K*^ mass 0. We define 6h as the angle between the 
direction of flight of the K and B in the K*'^ rest frame. 
The distribution of cos 6h is proportional to cos^ 6h for 
signal events and is expected to be flat for background 
events. We require | cos 6*^1 > 0.3. The charged kaons 
used to reconstruct the D'^ and K*'^ mesons are re- 
quired to satisfy kaon identification criteria, based on 
Cherenkov angle and dE / dx measurements and are typ- 
ically 85% efficient, depending on momentum and polar 
angle. Misidentification rates are at the 2% level. 

The B^ candidates are reconstructed by combining a 
and K*'^ candidate, constraining them to originate 
from a common vertex with a probability greater than 
0.1%. In forming the B, the D mass is constrained to its 
nominal value [J . The distribution of the cosine of the B 
polar angle with respect to the beam axis in the e+e" CM 
frame cos Ob is expected to be proportional to 1 — cos^ Ob- 
We require jcos^sl < 0.9. We measure two almost in- 
dependent kinematic variables: the beam-energy su bsti- 
tuted mass toes = \/ (£'o^/2 + po - phY/El -pb'^, and 
the energy difference Ai? = Eg — Eq /2, where E and p 
are energy and momentum, the subscripts B and refer 
to the candidate B and e"'"e~ system, respectively, and 
the asterisk denotes the e+e" CM frame. The distribu- 
tions of Toes and AE peak at the B mass and zero, re- 
spectively, for correctly reconstructed B mesons. The B 
candidates are required to have AE in the range [—16, 16] 
MeV (~ 1.3a), [-20,20] MeV (~ 1.5cr) and [-19,19] 
MeV (~ 1.4tT) for the Ktt, Ktttt'^ and Ktttttt modes, re- 
spectively. Finally we consider events with toes in the 
range [5.20,5.29] GeV/c^. 

We examine background B decays that have the same 
final state reconstructed particles as the signal decay to 
identify modes with peaking structure in toes or AE 
that can potentially mimic signal events. We iden- 
tify three such "peaking background" modes in the 
opposite-sign sample: 5° D~[K*'^K~]n+ (for Kn), 
B° -> D-[K*°K-]p+[t:+t:^] (for Ktttt^) and ^ 
D-[K*°K-]at[Tr+Tr+7T-] (for Ktttttt). To reduce their 
contribution we veto all candidates for which the invari- 
ant mass of the K*'^ and the K~ from the 1?° lies within 
6 MeV/c^ of the nominal D~ mass. 

After imposing the vetoes, the contributions of the 
peaking backgrounds to the Ktt, Ktttt^ and Ktttttt sam- 
ples are predicted to be less than 0.07, 0.05 and 0.12 
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events, respectively, at 95% probability. Other possi- 
ble sources of peaking background are i?" and 
[Z?°7r^]7r+, which contribute to the three de- 
cay modes in both the same- and opposite-sign samples. 
These events could be reconstructed as signal, due to 
misidentification of a tt as a if. We impose additional 
restrictions on the identification criteria of charged kaons 
from K* decays to reduce the contribution of these back- 
grounds to a negligible level. Charmless B decays, like 
5° K*^K'K, can also contribute. The number of ex- 
pected charmless background events, evaluated with data 
from the mass sidebands, is A^peofc=0.5 ± 0.5 (0.1 ± 
1.2) in the same (opposite) sign samples. 

In case of multiple D candidate (less than 1% of 
events), we choose the one with reconstructed D'^ mass 
closest to the nominal mass j7|. In the case of two B 
candidates reconstructed from the same ^ we choose 
the candidate with the largest value of | cos Oh \ ■ 

The overall reconstruction efficiencies for signal events 
are (13.2 ± 0.1)%, (5.2 ± 0.1)% and (6.5 ± 0.1)% for the 
Kit, Ktttt^ and Kmnr modes, respectively. 

After applying the selection criteria described above, 
the remaining background is composed of continuum 
events and combinatorial BB events. To discriminate 
against the continuum background events (the domi- 
nant background component), which, in contrast to BB 
events, have a jet-like shape, we use a Fisher discrimi- 
nant J- (Tsj . The discriminant is a linear combination 
of four variables calculated in the CM frame. The first 
discriminant variable is the cosine of the angle between 
the B thrust axis and the thrust axis of the rest of the 
event. The second and third variables are Lq = '^iPi, 
and L2 — ^jPijcos^ip, where the index i runs over 
all the reconstructed tracks and energy deposits in the 
calorimeter not associated with a track, the tracks and 
energy deposits used to reconstruct the B are excluded. 
Pi is the momentum, and 9i is the angle with respect to 
the thrust axis of the B candidate. The fourth variable 
is I At I, the absolute value of the measured proper time 
interval between the B and B decays, calculated from 
the measured separation between the decay points of the 
B and B along the beam direction. 

The coefficients of JF, chosen to maximize the sepa- 
ration between signal and continuum background, are 
determined using samples of simulated signal and con- 
tinuum events and validated using off-resonance data. 

The signal and background yields are extracted, sep- 
arately for each channel, by maximizing the extended 



likelihood C = (e-^')/(^0 ' N' 



,N 



nf=i/(x, 



Here Xj = {to£;5,JF}, is a set of parameters, N is the 
number of events in the selected sample and N' is the 
expectation value for the total number of events. The 



term /(x | 6,N') is defined as : 

RadsNdk' 



f{^\0,N')N'^ 
. Ndk 



1 + Rads 



fOS /^IflOS \ 
JSIGV^WSIG) 



1 



flfG{^\OflG) + 
n.ADS 



+ ^bkgfbkgi 



^hfeg ) + ^bkg fbk^g I ^ffeg ) 



(9) 



where Ndk* is the total number of signal events, Rads is 
the ratio between opposite- and same-sign signal events, 
and "bkg" refers to continuum or BB background, and 
Niit, Ng^t: and N^S are the number of same- 

and opposite-sign events for continuum and BB back- 
grounds. The probability density functions (PDFs) / 
are derived from MC and are defined as the product of 
one-dimensional distributions of mES and JF. The mEs 
distributions are modeled with a Gaussian for signal, and 
threshold functions with different parameters for the con- 
tinuum and BB backgrounds. The threshold function is 
expressed as follows: 



Aix) 



1_(_)2 
Xo 



(10) 



where xq represents the maximum allowed value for the 
variable x described by A{x) and c accounts for the shape 
of the distribution. The distributions are modeled with 
Gaussians. 

From the fit to data we extract Ne,k* , RadSi and the 
background yields {N^i^, 7V£f„ iV||, and N^^). We 
allow the mean of the signal vriES PDF and parameters 
of the continuum niES PDF's to float. 

The fitting procedure is validated using ensembles of 
simulated events. A large number of pseudo-experiments 
is generated with probability density functions and pa- 
rameters as obtained from the fit to the data. The fitting 
procedure is then performed on these samples. We find 
no bias on the number of fitted events for any of the 
components. 

The results for Ndk* , Rads and the background 
yields are summarized in Table IH The total num- 
ber of opposite-sign signal events in the three chan- 
nels is Ng^Q = 24.4l}Q g (statistical uncertainty only). 
Projections of the fit onto the variable vriEs are shown 



TABLE L Fit results for Ndk* , Rads and the number of 
background events, for the three channels. The uncertainties 
are statistical only. 



channel 


Ktv 


Ktvtt" 


A'tttttt 


Ndk* 


74± 12 


146 ± 17 


101 ± 17 


Rads 


W-UD' -0.054 




„ ,07+0.113 


N^- 

BB 


75± 16 


265 ± 33 


345 ± 35 




40± 17 


215 ±41 


327 ± 48 


' cont 


387 ± 22 


2497 ± 56 


2058 ± 53 


' cont 


1602 ± 41 


7793 ± 96 


6372 ± 91 
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in Fig. [2] for the opposite- and same-sign samples. To 
enhance the visibihty of the signal, events are required 
to satisfy T > 0.5 for Kt:, T > 0.7 for Kt:t:°, and T 
> 1 for Kmrn. These requirements have an efficiency of 
about 67%, 67% and 50% for signal and 9%, 5% and 3% 
for continuum background. 

The systematic uncertainties on Rads a-re summarized 
in Table |TT1 To evaluate the contributions related to the 
mES and T PDFs, we repeat the fit by varying all the 
PDF parameters that are fixed in the final fit within their 
statistical errors, as obtained from the paramctrization 
on simulated events. To evaluate the uncertainty arising 
from the assumption of negligible peaking background 
contributions, we repeat the fit by varying the number of 
these events within their statistical errors. In this evalu- 
ation, we consider all the possible sources of such back- 
grounds, coming from charmless B decays and from B de- 
cays with a D meson in the final state, as discussed above. 
For the multi-body D decays, the selection efficiency on 
same- and opposite-sign events has been confirmed to be 
the same, regardless of the difference in the Dalitz struc- 
ture, within a relative error of 3%. Finally, a systematic 
uncertainty associated with cross feed between same- and 
opposite-sign events is evaluated from MC studies to be 
(3.5 ± 0.5)%, (4.6 ± 0.6)% and (1.9 ± 0.4)% for the Xtt, 
Ktttt^ and Kmnr modes, respectively. The total system- 
atic uncertainties are defined by adding the individual 
terms in quadrature. 

TABLE II: Systematic uncertainties ARads, in units of 
[10-% for Rf^f and R^ZY ■ 



Source 


K-K 






Sig. PDF 


0.19 


0.11 


0.82 


Cent. PDF 


0.32 


0.02 


0.29 


BB PDF 


0.57 


0.16 


1.48 


Peaking bkg 


1.70 


0.87 


1.40 


(-CF I^DCS 




0.17 


0.39 


cross-feed 


0.04 


0.05 


0.02 


TOTAL 


1.8 


0.91 


2.2 



The final likelihood C{Rads) for each decay mode is 
obtained by convolving the likelihood returned by the 
fit with a Gaussian whose width equals the systematic 
uncertainty. Figure [3] shows C{Rads) for all three chan- 
nels, where we exclude the unphysical region Rads < 0. 
The integral of the likelihood corresponding to Rads < 
is 9.5% for Kn, 15.8% for Ktttt^ and 5.5% for Kmrn. 
The significance of observing a signal is evaluated in each 
channel using the ratio log(£rnax/'CQ), where £niax and 
Cq are the maximum likelihood values obtained from the 
nominal fit and from a fit in which the signal component 
is fixed to zero, respectively. We observe a ratio Rads 
different from zero with a significance of 1.1, 1.7 and 1.4 
standard deviations for the Kir, Kt:it^ and Kmnr modes, 
respectively. Since the measurements for the Rads ra- 
tios are not statistically significant, we calculate 95% 



probability limits by integrating the likelihoods, start- 
ing from Rads = 0. We obtain RADsiKir) < 0.244, 
RADsiKTTTT^) < 0.181 and RADsiK-KTnr) < 0.391 at 
95% probability. The overall significance of observing 
an Rads signal, evaluated from the combination of the 
three measurements, is 2.5 standard deviations. 

Following a Bayesian approach, the measurements of 
the Rads ratios are translated into a likelihood for rs- 
A large number of simulated experiments for the pa- 
rameters on which Rads depends (see Eq. [S]) are per- 
formed. For each experiment, the values of RADsiKn), 
RADsiKmr^) and Rads{Kiti:i:) are obtained and a 
weight C{RADs{Kn))C{RADs{Knn''))C{RADs{Knnn)) 
is computed. In the extraction procedure to determine 
rs^ we use the experimental distributions for the ra- 
tios, 5d{Kit), kDjKmr^), SoiKTrn^), kDiKmrTr) and 
SoiK-KTr-K) [3, [m, ■ All the remaining phases are ex- 
tracted from a flat distribution in the range [0, 27r]. rs is 
extracted from a flat distribution in the range [0, 1] and 
k is extracted from a Gaussian distribution with mean 
0.95 and standard deviation 0.03. We obtain the likeli- 
hood C{rs) shown in Fig.[51 The most probable value is 
rs — 0.26 and we obtain, by integrating the likelihood, 
the following 68% and 95% probability regions: 

rs £ [0.18,0.34] @ 68% probability, 
rs e [0.07,0.41] @ 95% probability. 

Given the functional dependence of Rads on rs 
{Rads ^ ''"s), the likehhoods corresponding to Rads < 
have no effective role in the extraction of rs ■ The de- 
pendence of the rs likelihood shown in Fig. 3] on the 
choice of the prior distributions in the extraction proce- 
dure has been studied. While the 68% and 95% prob- 
ability regions are quite stable, the likelihood shows a 
dependence on the choice of the prior distribution for val- 
ues of rs close to zero. For this reason, the region near 
zero should not be used to evaluate the significance. The 
significance to observe rs different from zero corresponds 
to the significance for RadSi and is evaluated from the 
combined fit to be 2.5 standard deviations. The result 
obtained for rs with the procedure described above is 
consistent with the result found from a direct fit to data 
assuming the simplified expression Rads = f's- 

In summary, we have presented a search for b u 
transitions in B'^ — > D^K*^ decays, analysed through an 
ADS method. We see indications of a signal at the level of 
2.5 standard deviations including systematic uncertain- 
ties. The most probable value for rs extracted from this 
result is rs = 0.26, where the 68% and 95% probability 
regions are indicated above. This result is in agreement 
with the phenomenological expectations from Ref. p^ . 
and shows that the use of these decays and related ones 
for the determination of 7 is interesting in present and 
future facilities. 

We are grateful for the extraordinary contributions of 
our PEP-II colleagues in achieving the excellent luminos- 
ity and machine conditions that have made this work pos- 
sible. The success of this project also relies critically on 
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FIG. 2: Projections of the fit onto the variable tties after a cut on T is applied (> 0.5 for Ktv, > 0.7 for Ktv-k'^ and > 1 
for K-ktvtt), to enhance the signal. The plots are shown for Kn (left), K-irn" (middle) and Ktvtttv (right), same-sign (top) and 
opposite-sign (bottom) events. The points with error bars are data. The dashed, dotted and dash-dotted lines represent the 
signal, continuum background and BB background contributions, respectively. The solid line represents the sum of all the 
contributions. 
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FIG. 3: Likelihood function for RADs(Kn) (left), RADs{Knn") (middle) and Rads{K tt-ktt) (right), for Rads > 0, thus 
excluding unphysical values. The dark and light shaded zones represent the 68% and 95% probability regions, respectively. 
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FIG. 4: Likelihood function for rs from the combination of 
the measurements of Rads obtained in the three D decay 
channels. The dark and light shaded zones represent the 68% 
and 95% probability regions, respectively. 
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